Intelligent data center

ABSTRACT

Provided, in one aspect, is a data center. The data center, in this aspect, includes a data center enclosure, the data center enclosure designed for a given supply of power (P s ). The data center, according to this aspect, further includes N independent coolable clusters of data center racks located within the data center enclosure, wherein N is at least two, and further wherein the N independent coolable clusters each have an ostensible power demand (P os ) approximately equal to P s /N, and each of the N independent coolable clusters has a respective actual power demand (P ac ) adjustable at, above or below the ostensible power demand (P os ).

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional Application Ser.No. 62/945,205, filed on Dec. 8, 2019, entitled “GPU DATA CENTER,”commonly assigned with this application and incorporated herein byreference.

TECHNICAL FIELD

At least one embodiment of the disclosure is directed, in general, todata centers and, more specifically, to designing, manufacturing andemploying fungible data centers for use in a variety of power (e.g.,high, low and everything in between) applications.

BACKGROUND

Many organizations use large scale computing facilities, such as datacenters, in their business. These data centers traditionally includelarge unencumbered rooms full of dozens of data center rack enclosures,each data center rack enclosure housing different electronic components,including the processors/data servers, network equipment, and computerequipment necessary to process, store, and exchange data as needed tocarry out an organization's operations. Unfortunately, today's datacenters are not designed to efficiently handle the ever increasing andcontinually changing power demand of the electronic components locatedwithin the data center rack enclosures.

BRIEF DESCRIPTION

Reference is now made to the following descriptions taken in conjunctionwith the accompanying drawings, in which:

FIG. 1 illustrates a data center designed, manufactured and operated toone or more embodiments of the disclosure;

FIGS. 2-4 illustrate power distribution units (PDUs) designed,manufactured and operated according to one or more alternativeembodiments of the disclosure; and

FIG. 5 illustrates a data center rack designed, manufactured andoperated according to one or more alternative embodiments of thedisclosure.

DETAILED DESCRIPTION

Traditional data centers are homogenous in nature, meaning that a singleunencumbered data center enclosure is full of dozens upon dozens (if nothundreds upon hundreds) of similarly, if not identically, configureddata center racks of electronic equipment. For instance, each of theracks might include a fixed number of processing equipment (e.g., dataservers), storage equipment and network equipment, likely configured ina desired manner within the rack, which is commonly reproduced time andtime again across many, if not all, of the data center racks within thedata center enclosure. Given the foregoing, each of the data centerracks within the traditional data center enclosure has very similar, ifnot identical, power requirements and heat dissipation requirements.

Accordingly, traditional data centers tend to be designed such that eachdata center rack within the data center enclosure equally shares theresources of the data center. Specifically, traditional data centers aredesigned for a given supply of power (P_(s)) and a given coolingcapacity (CC), and each data center rack within the data center receivesits approximately equal allotment of the given supply of power (P_(s))and the given cooling capacity (CC). As a non-limiting example, onetraditional data center might include 128 operable data center racks, at2× redundancy. Thus, in the traditional sense, each of the 128 operabledata center racks would receive its approximately equal 1/128^(th) ofthe given supply of power (P_(s)) and its approximately equal 1/128^(th)of the given cooling capacity (CC), or at least very close thereto.

It is envisioned that data centers will become less repetitive overtime, and thus certain data center racks and/or collections of datacenter racks will require more (or less) than their approximately equalallotment of the given supply of power (P_(s)) and/or given coolingcapacity (CC), than say another data center rack or collection of datacenter racks within the same data center enclosure. To accommodate theseever increasing requirements, all the while manufacturing a data centerthat has a suitable lifespan, traditional data center were over designedto accommodate the highest conceivable power and/or cooling capacityneeded. Unfortunately, over designed data centers are not only expensiveto manufacture, but in the general use case, are incredibly inefficientand expensive to operate—as much of the given supply of power (P_(s))and given cooling capacity (CC) goes unused. In fact, studies have shownthat over designed data centers leave a significant amount of theresources stranded.

As such, the present disclosure seeks to protect a fungible data centerdesign, wherein the fixed resources of the data center (e.g., includingwithout limitation power, cooling capacity, etc.), which weretraditionally approximately equally allotted amongst the data centerracks within the data center enclosure, may be moved (or at leastpartially moved) from one data center rack to another, or alternativelyfrom one cluster of data center racks to another cluster of data centerracks. In at least one embodiment, such a movement of the fixedresources of the data center amongst data center racks or clusters ofdata center racks may be performed without little and/or any real timephysical reconfiguration of the data center design.

Accordingly, a data center according to at least one embodiment of thedisclosure includes multiple degrees of freedom (e.g., three degrees offreedom in one example). For example, a data center designed,manufactured and operated according to the disclosure could include theability to vary the given supply of power (P_(s)) and/or given coolingcapacity (CC) of the data center enclosure as a whole, the ability tovary the allotted respective amounts of the given supply of power(P_(s)) and/or given cooling capacity (CC) amongst clusters of datacenter racks in relation to other clusters of data center racks, and theability to vary the allotted respective amounts of the given supply ofpower (P_(s)) and/or given cooling capacity (CC) amongst individualracks (e.g., whether within a given cluster of racks or outside a givencluster of racks) in relation to other individual data center racks.

Turning to FIG. 1, illustrated is a data center 100 designed,manufactured and operated according to one or more embodiments of thedisclosure. The data center 100 includes many of the benefits, includingthe fungible nature and the many degrees of freedom, as discussed above.The data center 100, in the illustrated embodiment, includes a datacenter enclosure 110. The data center enclosure 110, in one embodiment,is a single floor of a building. In an alternative embodiment, the datacenter enclosure 110 is a portion of a single floor of a building, oralternatively two or more floors of a building. In yet anotherembodiment, the data center enclosure 110 is a mobile data center.Notwithstanding the foregoing, a data center enclosure 110 according tothe present disclosure may be designed for a given supply of power(P_(s)) and/or given cooling capacity (CC).

In at least one embodiment, the given supply of power (P_(s)) enters thedata center enclosure 110 and goes into a power distribution element120. The power distribution element 120 may be any element and/orcomponent that helps distribute the given supply of power (P_(s)) of thedata center 100 to the various electronics within the data centerenclosure 110. In certain embodiments, the power distribution element120 includes a plurality of adjustable circuit breakers (e.g., anadjustable circuit breaker dedicated for each of the N independentlycoolable clusters, for example for limiting the actual power demand(P_(ac)) of one of the N independently coolable clusters to assure thatit is below the ostensible power demand (P_(os)), while not limitinganother of the N independently coolable clusters to allow it to be abovethe ostensible power demand (P_(os)), to keep a sum of the actual powerdemands (P_(ac)) for the N independent coolable clusters at or below thegiven supply of power (P_(s))). In at least one embodiment, the givencooling capacity (CC) enters the data center enclosure 110 and goes intoa cooling distribution element 130. The cooling distribution element 130may be any element and/or component that helps distribute the givencooling capacity (CC) to the various electronics within the data centerenclosure 110. While the cooling distribution element 130 is illustratedat the data center enclosure 110 level, other embodiments may existwherein multiple cooling distribution elements 130 are located withinthe data center enclosure (e.g., a cooling distribution element 130dedicated for each of the N independently coolable clusters).Furthermore, the cooling distribution element 130 is not limited to onlydistributing the product of air-based cooling systems, but could also beused to distribute the product of liquid-based cooling systems,including cold plate single phase cooling systems, cold plate two phasecooling systems, or immersion cooling systems. Furthermore, the coolingdistribution element 130 is not limited to distributing the product ofany single type of cooling system, but could be used to distribute theproduct of multiple types of cooling systems, including those systemsdescribed above. The cooling distribution element 130 is capable ofsophisticated control of the coolants, independently or concurrently, inthe data center enclosure 110. For instance, the cooling distributionelement 130 may be adapted to control the temperature, pressure or flowrate, among others, so that the coolant(s) is appropriately distributedto extract heat generated within the data center racks of the datacenter enclosure 110.

In at least one embodiment, the data center enclosure 110 includes aplurality of data center racks 140. The specific number of data centerracks 140 within a given data center enclosure 110 may vary greatly fromdata center 100 design to data center design 100, as well as theapplication of the data center 100. Nevertheless, typical data centers100 include dozens upon dozens of data center racks 140, if not morethan a hundred data center racks 140.

In accordance with at least one embodiment, the plurality of data centerracks 140 are separated into N independently coolable clusters 150 ofdata center racks 140. In the embodiment of FIG. 1, the plurality ofdata center racks 140 have been separated into eight (8) independentlycoolable clusters 150 of data center racks 140 (e.g., clusters 1-8), forexample by placing the eight (8) independently coolable clusters 150 ineight (8) separate enclosures and/or rooms 155. Other embodiments areenvisioned where N is at least four (4), and each of the at least four(4) independent coolable clusters 150 includes at least 8 data centerracks 140, among other configurations. The enclosures and/or rooms 155may take on many different configurations and remain within the scope ofthe disclosure. What optimally results, in at least one embodiment, ispartial and/or complete local thermal isolation of the eight (8)independently coolable clusters 150 from one another.

In at least one embodiment, the eight (8) independently coolableclusters 150 may be arranged into pairs. In the illustrated embodiment,what results are four (4) separate Quads (e.g., A, B, C and D). In atleast one embodiment, the four (4) separate Quads are four (4)separately controllable Quads. In at least one embodiment, Quad Aincludes the first and second (e.g., 1 and 2) independently coolableclusters 150, Quad B includes the third and fourth (e.g., 3 and 4)independently coolable clusters 150, Quad C includes the fifth and sixth(e.g., 5 and 6) independently coolable clusters 150, and Quad D includesthe seventh and eighth (e.g., 7 and 8) independently coolable clusters150.

Each of the data center racks 140, in accordance with at least oneembodiment, include various different types and amounts of electronicequipment requiring power, including any number of various differenttypes of processing equipment (e.g., also referred to as data servers),storage equipment, network equipment and power distribution unit(s),among others. In at least one embodiment, the various different typesand amounts of electronic equipment may be used to provide the FloatingPoint Operations per Second (FLOPS) needed for high performancecomputing, such as may be necessary for artificial intelligence (AI)applications. In at least one embodiment, the various different typesand amounts of electronic equipment can be used to provide the storageand networking needed to support the large-scale deep neural network(DNN) training that powers software development for autonomous vehicles,internal AI for companies, and robotics development, among others. In atleast one embodiment, the various different types and amounts ofelectronic equipment can be used with reactive machines, autonomousmachines, self-aware machines, and self-learning machines that allrequire a massive compute intensive server infrastructure and computingpower.

In at least one embodiment, the processing equipment can be parallelprocessing units, such as graphics processing units (GPUs), or serialprocessing units, such as central processing units (CPUs). In at leastone embodiment, the processing equipment can also be other types ofcircuits having at least a portion of the processing capabilities of aGPU or CPU. In at least one embodiment, the processing equipment can be,for example, application-specific integrated circuits (ASICs), digitalsignal processors (DSPs), or programmable logic devices such asprogrammable array logic (PAL), generic array logic (GAL), fieldprogrammable gate arrays (FPGA), or another type of computer processingdevice (CPD). In at least one embodiment, the data center racks 140 caninclude a single type of processing equipment or a combination of any ofdifferent types of processing equipment. In at least one embodiment, theprocessing equipment can include an analog processor.

In at least one embodiment, the processing equipment can be high-density(HD) GPU equipment that include storage nodes and high performance GPUcompute nodes designed to accelerate deep learning applications. In atleast one embodiment, the high performance GPU compute nodes can beprocessing equipment designed for general-purpose computing on graphicsprocessing units (GPUs) to accelerate deep learning applications. In atleast one embodiment, the GPU compute nodes can be processing equipmentof the DGX or EGX product lines from NVIDIA Corporation of Santa Clara,Calif. In at least one embodiment, a version of the DGX product line,DGX-2, is used herein as one example of a GPU compute node.

In at least one embodiment, the various different types and amounts ofelectronic equipment needed for data centers to provide the amount ofcomputing power being requested require a tremendous amount of power,both for the data center 100 as a whole, and on a rack-by-rack basis140. Accordingly, each of the data center racks 140 may include a powerdistribution unit to distribute this tremendous amount of power. In atleast one embodiment, while low-power power distribution units may besufficient for data center racks having a power demand up toapproximately 15 kVA, data center racks having a higher power demand(e.g., above 15 kVA) require power distribution units capable ofproviding more power. In at least one embodiment, one or more of thedata center racks 140 may include novel power distribution units capableof handling the higher power demands encountered with certain datacenter racks, regardless of the processor type located therein.

In at least one embodiment, the data center 100 additionally includesone or more R&D clusters 160. In the illustrated embodiment, the datacenter 100 includes a pair of R&D clusters 160. The R&D clusters 160 areseparate high power density clusters for testing of future air-cooledand/or liquid-to-chip-cooled servers.

The N independent coolable clusters 150 (e.g., the eight (8) independentcoolable clusters 150 illustrated in FIG. 1), in at least oneembodiment, each have an ostensible power demand (P_(os)) approximatelyequal to the given power supply P_(s) divided by the number N, orP_(s)/N. Typically, each of the N independent coolable clusters 150 willhave the same ostensible power demand (P_(os)). Further to thisembodiment, each of the N independent coolable clusters 150 may have arespective actual power demand (P_(ac)), which may change over time. Therespective actual power demand (P_(ac)), in at least on embodiment, isapproximately equal to the actual power consumed (e.g., for a givenperiod of time) for each of the independent coolable clusters 150. In atleast one embodiment, the respective actual power demands (P_(ac)) areadjustable at, above or below the ostensible power demand (P_(os)), forexample by placing additional electronic equipment within the datacenter racks 140, or alternatively taxing the existing electronicequipment at greater levels.

In accordance with at least one other embodiment, the N independentcoolable clusters 150 are configured such that when a first of the Nindependent coolable clusters (e.g., say the first coolable cluster 150)has its actual power demand (P_(ac)) above the ostensible power demand(P_(os)), a second of the N independent coolable clusters 150 (e.g., saythe second coolable cluster 150) has its actual power demand (P_(ac))below the ostensible power demand (P_(os)). Accordingly, in at leastthis embodiment, a sum of the actual power demands (P_(ac)) for the Nindependent coolable clusters 150 is maintained at or below the givensupply of power (P_(s)). Thus, according to this embodiment, additionalneeded power for one of the N independently coolable clusters 150 may beharvested and/or collected from an excess (e.g., unused or strandedpower) from another of the N independently coolable clusters 150. Inthis sense, the N independently coolable clusters 150 may be operated ina ping-pong like nature, wherein at certain times ones of the Nindependently coolable clusters 150 are operated with their actual powerdemand (P_(ac)) above the ostensible power demand (P_(os)), while at thesame time certain other ones of the ones of the N independently coolableclusters 150 are operated with their actual power demand (P_(ac)) belowthe ostensible power demand (P_(os)), this occurring without exceedingthe given supply of power (P_(s)).

The ability to shift the allotted amount of the given supply of power(P_(s)) amongst the various N independently coolable clusters 150 toaccommodate the varying actual power demand (P_(ac)) of the various Nindependently coolable clusters 150 is not so easy as to just plug inmore electronic components into a given independently coolable cluster150 and go. For instance, the data center enclosure 110 is designed fora given cooling capacity (CC), and thus the N independent coolableclusters each have a respective ostensible cooling capacity (CC_(os))approximately equal to given cooling capacity (CC) divided by N, orCC/N. However, the instant data center 100 may be designed such thatwhen the actual power demand (P_(ac)) of a first of the N independentcoolable clusters 150 is increased above its ostensible power demand(P_(os)), its actual cooling capacity (CC_(ac)) may be increased aboveits ostensible cooling capacity (CC_(os)). Moreover, the instant datacenter 100 may optionally be designed such that when the actual powerdemand (P_(ac)) of a second of the N independent coolable clusters 150is decreased below its ostensible power demand (P_(os)), its actualcooling capacity (CC_(ac)) may be decreased below its ostensible coolingcapacity (CC_(os)).

Advantageously, in at least one embodiment, the N clusters 150 areindependently coolable from one another, and thus the actual coolingcapacity (CC_(ac)) of each of the N independently coolable clusters 150may also be independently controlled. In at least one embodiment, thedata center enclosure 110 includes a raised floor for providing airflowto the N independent coolable clusters 150, thereby assisting in theindependent control of the N independently coolable clusters 150. In atleast one other embodiment, at least one of a temperature, an airflowand a pressure is independently adjustable across the N independentcoolable clusters of data center racks. In certain embodiments, just thetemperature is independently adjustable, but in other embodiments justthe airflow or the pressure is independently adjustable. In yet otherembodiments, any combination of the temperature, airflow and pressure isindependently adjustable. While the N clusters 150 have been discussedas being independently coolable, in certain embodiments the data centerracks 140 within a given cluster 150 may be independently cooled.

Moreover, in at least one other embodiment, the N independent coolableclusters 150 each include a multimode cooling system, the multimodecooling system providing the ability to increase or decrease therespective actual cooling capacities (CC_(ac)) of the N independentcoolable clusters above or below their ostensible cooling capacities(CC_(os)). In at least one embodiment, the multimode cooling system ofeach of the N independent coolable clusters 150 comprises an air-basedcooling subsystem and a liquid-based cooling subsystem. In such anembodiment, at least one of the temperature, the airflow or the pressuremay be adjusted for the air-based cooling subsystem, while thetemperature and flow rate may be adjusted for the liquid-based coolingsubsystem. In yet another embodiment, the multimode cooling system ofeach of the N independent coolable clusters 150 comprises two or more ofthe air-based cooling subsystem, the liquid-based cooling subsystem(e.g., immersion cooling subsystem), a phase-based cooling subsystem(e.g., a cold plate single phase cooling subsystem, cold plate two phasecooling subsystem, etc.), or a dielectric-based cooling subsystem. Theliquid-based cooling subsystem may require independent supply (e.g.,colder) and return (e.g., warmer) manifolds built into the N independentcoolable clusters 150, as well as part of the data center racksthemselves. Such independent supply (e.g., colder) and return (e.g.,warmer) manifolds may include quick connects, thereby supplying the sameto the N independent coolable clusters 150 and the data center racksthemselves. In certain embodiments, every single rack within a givencoolable cluster 150 is pre-configured for both the air-based coolingsubsystem and the liquid-based cooling subsystem.

The multimode cooling systems of each of the N independently coolableclusters 150 may, therefore, be rated for different cooling capacitiesas each of the subsystems may have a different rated cooling capacity.In at least one embodiment, the air-based cooling subsystem is rated forbetween 10 kilowatt (KW or kW) to 57 KW of generated heat. Similarly,the liquid-based cooling subsystem, the phase-based cooling subsystem,and the dielectric-based cooling subsystem could be rated for between 30KW and 120 KW of generated heat. As such, the multiple mode coolingsystem enables switching of different cooling system, which may be usedfor peak shaving and for removal of regular to extreme heat due toeither computational loads or environmental conditions.

The ability to easily accommodate the various different amounts of heatgenerated amongst the N independently coolable clusters 150 is not theonly requirement to be able to increasing and/or decreasing the actualpower demand (P_(ac)) amongst the N independently coolable clusters 150.It is additionally necessary to distribute the increased actual powerdemand (P_(ac)) amongst the data center racks within each of the Nindependently coolable clusters 150. Accordingly, each of the datacenter racks within each of the N independently coolable clusters 150would traditionally include one or more power distribution units.Traditional power distribution units are not capable of accommodatingthe increased actual power demand (P_(ac)) for the data center racks ofthe N independently coolable clusters 150. Thus, in accordance with oneembodiment, certain ones of the data center racks within certain ones ofthe N independently coolable clusters 150 (e.g., each of the data centerracks within each of the N independently coolable clusters 150) includeone or more high power, power distribution units. In at least oneembodiment, each of the high power, power distribution units includes apower distribution unit enclosure having a connector surface, one ormore low power inlet connectors extending through the connector surface,and one or more higher power inlet connectors extending through theconnector surface, wherein a ratio of a number of the one or more higherpower inlet connectors to a number of the one or more low power inletconnectors is at least 1:1.

Given the foregoing, in one non-limiting use based example, the datacenter 100 might have a given supply of power (P_(s)) of approximately4.5 MW. Given that the data center 100 illustrated in FIG. 1 includeseight (8) independently coolable clusters 150, each of the eight (8)coolable clusters 150 would have an ostensible power demand (P_(os))approximately equal to 4.5 MW divided by eight (8), or approximately 560kW. Likewise, in this non-limiting use-based example, each of the datacenter racks 140 might have an ostensible power demand (P_(os)) ofapproximately 35 kW. In accordance with the disclosure, and employingthe cooling distribution and/or power distribution ideas discussedabove, certain ones of the eight (8) independently coolable cluster 150could be increased above their an ostensible power demand (P_(os)) ofapproximately 560 kW. For instance, four (4) of the eight (8)independently coolable cluster 150 could have their respective actualpower demand (P_(ac)) adjusted to zero (e.g., they are not inoperation), and the other of the four (4) independently coolable cluster150 could have their respective actual power demand (P_(ac)) doubled toapproximately 1120 kW each, all the while not exceeding given supply ofpower (P_(s)) of 4.5 MW. In another embodiment, four (4) of the eight(8) independently coolable cluster 150 could have their respectiveactual power demand (P_(ac)) adjusted to 360 kW, and the other of thefour (4) independently coolable cluster 150 could have their respectiveactual power demand (P_(ac)) increased to approximately 760 kW each, allthe while not exceeding given supply of power (P_(s)) of 4.5 MW.Moreover, any combination of increased actual power demand (P_(ac)) anddecreased actual power demand (P_(ac)) among the eight (8) independentlycoolable clusters 150, so long as the sum thereof does not exceed 4.5MW, could be employed. Similarly, it is envisioned that the actual powerdemand (P_(ac)) of the individual data center racks within each of theeight (8) independently coolable clusters 150, could also be increasedand/or decreased, so long as the given supply of power (P_(s)) does notexceed 4.5 MW (e.g., in this use based example).

FIG. 2 illustrates a power distribution unit 200 designed, manufacturedand operated according to at least one embodiment of the disclosure. Thepower distribution unit 200 could be used as the one or more powerdistribution units employed in the data center racks 140 of FIG. 1, andgreatly assists with the above-discussed fungible nature of the datacenter 100. In at least one embodiment, the power distribution unit 200includes a power distribution unit enclosure 210. In at least oneembodiment, the power distribution unit enclosure 210 includes aconnector surface 220, a rear surface 225, and one or more side surfaces230. In at least one embodiment, the connector surface 220 and rearsurface 225 are defined by the width (w) and height (h), whereas the oneor more side surfaces are defined by the height (h) and depth (d).

In at least one embodiment, the width (w), height (h) and depth (d) ofthe power distribution unit enclosure 210 may vary based upon the designof the power distribution unit 200. In at least one embodiment,nevertheless, the width (w) of the power distribution unit enclosure 210ranges from approximately 405 mm (e.g., approximately 16 inches) toapproximately 445 mm (e.g., approximately 17.5 inches), as might be usedwith an approximately 483 mm (e.g., approximately 19 inch) data centerrack. In at least one embodiment, the width (w) of the powerdistribution unit enclosure 210 ranges from approximately 508 mm (e.g.,approximately 20 inches) to approximately 545 mm (e.g., approximately21.5 inches), as might be used with an approximately 584 mm (e.g.,approximately 23 inch) data center rack. In at least one embodiment,such widths (w) are consistent with rack widths (w_(r)) of certainstandard data center racks. In at least one embodiment, variousdifferent spacers and rack mount ears may also be used to accommodatedifferent power distribution unit 200 widths (w).

In at least one embodiment, the height (h) of the power distributionunit enclosure 210 illustrated in FIG. 2 is described based upon anumber of rack units (U). In at least one embodiment, a rack unit (U),as that term is used in the present disclosure, is equal toapproximately 44.5 mm (e.g., approximately 1.75 inches). In at least oneembodiment, a 1 U height (h) is equal to approximately 44.5 mm (e.g.,approximately 1.75 inches), a 2 U height (h) is equal to approximately89 mm (e.g., approximately 3.5 inches), a 3 U height (h) is equal toapproximately 133.5 mm (e.g., approximately 5.25 inches), etc. In atleast one embodiment, including the illustrated embodiment of FIG. 2,the power distribution unit enclosure 210 has a 2 U height (h).

In at least one embodiment, the depth (d) of the power distribution unitenclosure 210 illustrated in FIG. 2 is based upon the size of theinternal components that make up the power distribution unit 200. In atleast one embodiment, the depth (d) of the power distribution unitenclosure 210 is generally less than the rack depth (d_(r)) of the datacenter rack it is housed within, and thus in at least one embodimentless than approximately 1016 mm (e.g., approximately 40 inches), or lessthan approximately 915 mm (e.g., approximately 36 inches), depending onthe data center rack chosen. In at least one embodiment, including theembodiment of FIG. 2, the width (w) and depth (d) are each greater thanthe height (h).

In at least one embodiment, rack mount ears 240 are coupled to the powerdistribution unit enclosure 210. In at least one embodiment, the rackmount ears 240 may include a collection of appropriately spaced holes,which align with same spaced holes in a rail system of a data centerrack, for coupling the power distribution unit 200 to the data centerrack. In at least one embodiment, the rack mount ears 240 may beremovably coupled to the power distribution unit enclosure 210, such asshown in FIG. 2. In at least one embodiment, the rack mount ears 240 mayalternatively be fixedly coupled to the power distribution unitenclosure 210. In at least one embodiment, including the embodiment ofFIG. 2, the power distribution unit enclosure 210 is a 2U rack mountpower distribution unit enclosure.

In at least one embodiment, including the illustrated embodiment of FIG.2, the power distribution unit 200 includes one or more low power inletconnectors 250 and one or more higher power inlet connectors 260, eachextending through the connector surface 220. In at least one embodiment,the phrases “low power” and “higher power,” as used herein with respectto the connectors, are relative terms based upon their maximum currentcarrying capacity. In at least one embodiment, the one or more low powerinlet connectors 250 accordingly have a lower maximum current carryingcapacity than the one or more higher power inlet connectors 260, in manyexamples by 10 percent or more.

In at least one embodiment, the one or more low power inlet connectors250 have an ampacity of about 16 amps or less. In at least oneembodiment, Ampacity, as that term is used herein, is defined as themaximum current, in amperes, that a conductor can carry continuouslyunder the conditions of use without exceeding its temperature rating(e.g., maximum pin temperature), as calculated by the National ElectricCode. In at least one embodiment, such as the embodiment of FIG. 2, theone or more low power inlet connectors 250 are one or more C13 inletconnectors. In at least one embodiment, for example in the UnitedStates, C13 inlet connectors have an ampacity of about 16 amps, for a 70degrees Celsius maximum pin temperature (e.g., 120V×60 Hz). In at leastone embodiment, for example in APAC or EU, C13 inlet connectors have anampacity of about 10 amps, for a 70 degrees Celsius maximum pintemperature (e.g., 230V×50 Hz). In at least one embodiment, the one ormore C13 inlet connectors are configured to couple to or engage with anelectrical power cord having a C14 end, both of which may commonly befound associated with personal computers and related peripherals.

In at least one embodiment, the one or more higher power inletconnectors 260 have an ampacity of at least about 20 amps. In at leastone embodiment, such as the embodiment of FIG. 2, the one or more higherpower inlet connectors 260 are one or more C19 inlet connectors. In atleast one embodiment, such as in the United States, C19 inlet connectorshave an ampacity of about 20 amps, for a 70 degrees Celsius maximum pintemperature (e.g., 120V×60 Hz). In at least one embodiment, such as inAPAC or EU, C19 inlet connectors have an ampacity of about 16 amps, fora 70 degrees Celsius maximum pin temperature (e.g., 230V×50 Hz). In atleast one embodiment, the one or more C19 inlet connectors areconfigured to couple to or engage with an electrical power cord having aC20 end. In at least one embodiment, C19 inlet connectors and C20 endsare common for supplying power to enterprise-class servers,uninterruptable power supplies (UPS), datacenter rack-mountedpower-distribution units and other equipment that draw too much currentfor C13/C14 types.

In at least one embodiment, an exact number of low power inletconnectors 250 and higher power inlet connectors 260 is based upon thepower demand placed upon the power distribution unit 200 designed,manufactured and operated according to the disclosure. In at least oneembodiment, in contrast to existing power distribution units, the powerdistribution unit 200 is designed such that a ratio of a number of theone or more higher power inlet connectors 260 to a number of the one ormore low power inlet connectors 250 is at least 1:1. In at least oneembodiment, if there is only one low power inlet connector 250, therewill also be at least one higher power inlet connector 260. In at leastone embodiment, if there are only two low power inlet connectors 250,there will similarly be at least two higher power inlet connectors 260.

In at least one embodiment, the number of higher power inlet connectors260 greatly outweighs the number of low power inlet connectors 250. Inat least one embodiment, for instance, the ratio of the number of theone or more higher power inlet connectors 260 to the number of the oneor more low power inlet connectors 250 is at least 3:2. In at least oneembodiment, the ratio of the number of the one or more higher powerinlet connectors 260 to the number of the one or more low power inletconnectors 250 is at least 2:1. In at least one embodiment, the ratio ofthe number of the one or more higher power inlet connectors 260 to thenumber of the one or more low power inlet connectors 250 is at least3:1, or even 7:1 or greater. In at least one embodiment, such as thatillustrated in FIG. 2, the power distribution unit 200 includes twelve(12) higher power inlet connectors 260 (e.g., C19 inlet connectors) andsix (6) low power inlet connectors 250 (e.g., C13 inlet connectors),again within a 2 U height (h) of the power distribution unit enclosure210.

In at least one embodiment, the one or more low power inlet connectors250 and one or more higher power inlet connectors 260 that extendthrough the connector surface 220 collectively have a load powercapacity of at least about 17.3 kVA. In at least one embodiment, thephrase “load power capacity,” as used throughout this disclosure, refersto 80% of the maximum amount of power the power distribution unit 200 iscapable of supporting, which is a value chosen in North America in anattempt to avoid circuit overload and fire risk. In at least oneembodiment, a load power capacity of at least about 17.3 kVA, would inturn represent a maximum power capacity of at least about 21.6 kVA at 60amps.

In at least one embodiment, the one or more low power inlet connectors250 and one or more higher power inlet connectors 260 that extendthrough the connector surface 220 collectively have a load powercapacity of at least about 23 kVA (e.g., a maximum power capacity of atleast about 28.8 kVA at 80 amps), and in yet another embodiment of atleast about 28.8 kVA (e.g., a maximum power capacity of at least about36 kVA at 100 amps). In at least one embodiment, the one or more lowpower inlet connectors 250 and one or more higher power inlet connectors260 that extend through the connector surface 220 collectively have aload power capacity of at least about 34.5 kVA (e.g., a maximum powercapacity of at least about 43 kVA at 60 amps), and in yet anotherdifferent embodiment of at least about 57.5 kVA (e.g., a maximum powercapacity of at least about 71.8 kVA at 100 amps). In at least oneembodiment, a power distribution unit 200 designed, manufactured andoperated according to the disclosure desirably employs the most numberof higher power inlet connectors 260 having the highest load powercapacity within the smallest power distribution unit enclosure 210.

In at least one embodiment, the power distribution unit 200 additionallyincludes a main power interface 270. In at least one embodiment, themain power interface 270 is the point at which the power distributionunit 200 receives AC power from a power source. In at least oneembodiment, the main power interface 270 is coupled to a bus bar of adata center rack, for example using a main power cable. In at least oneembodiment, such as that illustrated in FIG. 2, the main power interface270 is on the right hand side of the back of the power distribution unit200 (e.g., as looking at a back thereof). In at least one embodiment,the main power interface 270 may be located on the left hand side of theback of the power distribution unit 200 (e.g., as looking at a backthereof), or alternatively the power distribution unit 200 may have theoption of placing the main power interface 270 on the right or left handside thereof. In at least one embodiment, it may be beneficial to employa right hand side main power interface 270 on a first power distributionunit 200 located within a server rack, and a left hand side main powerinterface 270 on a second power distribution unit 200 located within theserver rack, as doing such helps physically manage the routing of power.

In at least one embodiment, in addition to the power distribution unit200 distributing power amongst its one or more low power inletconnectors 250 and its one or more higher power inlet connectors 260,the power distribution unit 200 may have a transformer to step down thepower for certain other lower power devices. In at least one embodiment,the power distribution unit 200 may additionally have meteringcapabilities, including both input and output metering capabilities. Inat least one embodiment, the power distribution unit 200 mayadditionally be a managed power distribution unit, such that a user mayremotely manage its features, including inlet connector switching, andinlet connector level controlling and monitoring, among other remotelymanaged features.

FIG. 3 illustrates a power distribution unit 300 designed, manufacturedand operated according to at least on embodiment of the disclosure. Inat least one embodiment, the power distribution unit 300 is similar inmany respects to the power distribution unit 200 described in detailabove. In at least one embodiment, like reference numbers have been usedto represent similar, if not identical, features. In at least oneembodiment, the power distribution unit 300 differs, for the most part,from the power distribution unit 200, in that the power distributionunit 300 has a height (h) of 3 U, and furthermore includes six (6) lowpower inlet connectors 350 and eighteen (18) higher power inletconnectors 360 extending through the connector surface 220. In at leastone embodiment, such as the embodiment of FIG. 3, the power distributionunit 300 has a load power capacity of at least about 34.5 kVA.

FIG. 4 illustrates a power distribution unit 400 designed, manufacturedand operated according to at least one other embodiment of thedisclosure. In at least one embodiment, the power distribution unit 400is similar in many respects to the power distribution units 200, 300described in detail above. In at least one embodiment, like referencenumbers have been used to represent similar, if not identical, features.In at least one embodiment, the power distribution unit 400 differs, forthe most part, from the power distribution units 200, 300, in that thepower distribution unit 400 includes one or more higher power inletconnectors 460 extending through the connector surface 220, but does notinclude any low power inlet connectors extending through the connectorsurface. In at least one embodiment, the higher power inlet connectors460 collectively have a load power capacity of at least about 17.3 kVA,if not at least about 34.5 kVA.

In at least one embodiment, the power distribution unit 400 additionallydiffers from the power distribution units 200, 300 in that it includestwenty-four (24) higher power inlet connectors 460 extending through theconnector surface 220. In at least one embodiment, including theembodiment of FIG. 4, each of the one or more higher power inletconnectors 460 may have a maximum current carrying capacity of at leastabout 16 amps, such is the case if the higher power inlet connectors 460were C19 inlet connectors.

In at least one embodiment, collections of different higher power inletconnectors 460 may be individually labeled, for example to align withdifferent phases of the power. In at least one embodiment, thecollection 470 a might be color coded for a first phase of power, whilecollection 470 b might be color coded for a second phase of power, andcollection 470 c might be color coded for a third phase of power. In atleast one embodiment, the first collection 470 a might be color codedred to indicate the first phase, the second collection 470 b might becolor coded black to indicate the second phase, and the third collection470 c might be color coded green to indicate the third phase. In atleast one embodiment, the color coding may be applied using paint,stickers, or another readily visible indicator, and may be used with anypower distribution unit manufactured according to the disclosure.

FIG. 5 illustrates a data center rack 500 designed, manufactured andoperated according to at least one embodiment of the present disclosure.The data center rack 500 may be similar to any one of the data centerracks 140 in the N independent coolable clusters 150, as described abovewith regard to FIG. 1, and remain within the scope of the disclosure. Inat least one embodiment, the data center rack 500 includes a rackenclosure 510. In at least one embodiment, the rack enclosure 510 may bea standard Electronic Industries Alliance (EIA) rack enclosure, amongothers, and remain within the purview of the disclosure. In at least oneembodiment, the rack enclosure 510 includes an enclosure 520, a railsystem 525 located within the enclosure 520, and an optional door 530.

In at least one embodiment, the rack enclosure 510 has a rack width(w_(r)), a rack height (h_(r)) and a rack depth (d_(r)) (notillustrated), which may be defined by the rail system 525. In at leastone embodiment, while not limited to such, the rack enclosure 510employs a width (w_(r)) of either approximately 483 mm (e.g.,approximately 19 inches) or approximately 584 mm (e.g., approximately 23inches). In at least one embodiment, the rack enclosure 510 employs aheight (h_(r)) of approximately 1246 mm (e.g., 28 U—approximately 49inches), approximately 1778 mm (e.g., 40 U—approximately 70 inches),approximately 1867 mm (e.g., 42 U—approximately 73.5 inches),approximately 2000 mm (e.g., 45 U—approximately 78.75 inches), orapproximately 2134 mm (e.g., 48 U—approximately 84 inches). In at leastone embodiment, the rack height (h_(r)) is at least twenty-eight rackunits (28 U), and the rack width (w_(r)) is at least approximately 480mm. In at least one embodiment, the depth (d_(r)) of the data centerrack often varies, for example depending on the equipment that will behoused within the rack enclosure 510.

In at least one embodiment, including the embodiment of FIG. 5, one ormore power distribution units 540 are physically coupled to the rackenclosure 510. In at least one embodiment, a first power distributionunit 540 a and a second power distribution unit 540 b are physicallycoupled to the rack enclosure 510. In at least one embodiment, such asthe embodiment of FIG. 5, the power distribution units 540 a, 540 b arefixed within the rail system 525 using one or more fasteners 528. In atleast one embodiment, the power distribution units 540 a, 540 b may beany power distribution unit designed, manufactured and operatedaccording to the present disclosure. In at least one embodiment, thepower distribution units 540 a, 540 b are similar, if not identical, toone or the other of the power distribution units 200, 300, 400illustrated above with regard to FIGS. 2-4. In at least one embodiment,the power distribution units 540 a, 540 b would each include a powerdistribution unit enclosure having a connector surface, one or more lowpower inlet connectors extending through the connector surface, and oneor more higher power inlet connectors extending through the connectorsurface, as described above. In at least one embodiment, a ratio of thenumber of the one or more higher power inlet connectors to the number ofthe one or more low power inlet connectors for each of the powerdistribution units 540 a, 540 b is at least 1:1. In at least oneembodiment, the one or more low power inlet connectors and one or morehigher power inlet connectors, for each of the power distribution units540 a, 540 b, may collectively have a load power capacity of at leastabout 17.3 kVA.

In at least one embodiment, the power distribution units 540 a, 540 bare coupled to a main power source 550 using one or more main powercables 555, which may each in turn employ an IED 60309 plug at an end ofthe main power cable 555 opposite the power distribution units 540 a,540 b. In at least one embodiment, the main power source 550 is a busbar located within the enclosure 520. In at least one embodiment, themain power source 550 comprises a different feature than a bus bar. Inat least one embodiment, the main power cables 555 extend between themain power source 550 and a main power interface of the powerdistribution units 540 a, 540 b.

In at least one embodiment, the power distribution units 540 a, 540 bare positioned within about 25 percent of a vertical midpoint (e.g., asdefined by a midpoint of height (h_(r))) of the rail system 525. In atleast one embodiment, the power distribution units 540 a, 540 b arepositioned within about 10 percent of the vertical midpoint of the railsystem 525, and in even yet at least one other embodiment within about 5percent of the vertical midpoint of the rail system 525. In at least oneembodiment, such a position may allow the power distribution units 540a, 540 b to be substantially equal distance from any electronics aboveand below them in the rail system 525.

In at least one embodiment, the data center rack 500 additionallyincludes a first data server 560 a and a second data server 560 bphysically coupled to the rack enclosure 510. In at least oneembodiment, one or both of the first and second data servers 560 a, 560b may be CPU data servers. In at least one embodiment, one or both ofthe first and second data servers 560 a, 560 b may be GPU data servers.In at least one embodiment, one or both of the first and second dataservers 560 a, 560 b may be data servers that include other types ofprocessors, including application-specific integrated circuits (ASICs),digital signal processors (DSPs), or programmable logic devices such asprogrammable array logic (PAL), generic array logic (GAL), fieldprogrammable gate arrays (FPGA), or another type of computer processingdevice (CPD). In at least one embodiment, the first and second dataservers 560 a, 560 b, may be a collection of CPU, GPU, and other dataservers including the aforementioned processors. In at least oneembodiment, one or both of the first and second data servers 560 a, 560b may be GPU data servers from the DGX product line from NVIDIA. In atleast one embodiment, including the illustrated embodiment of FIG. 5,both the first and second data servers 560 a, 560 b are DGX-2 GPU dataservers, as might be obtained from NVIDIA. In at least one embodiment,unless otherwise indicated, the present disclosure should not be limitedto CPU or GPU data servers, and moreover should not be limited to anyspecific manufacturer thereof.

In at least one embodiment, including in the embodiment of FIG. 5, thefirst and second data servers 560 a, 560 b are fixed within the railsystem 525 using the one or more fasteners 528. In at least oneembodiment, the first and second data servers 560 a, 560 b areadditionally electrically coupled to the power distribution units 540 a,540 b. In at least one embodiment, one or more server power cords 570couple higher power inlet connectors of the power distribution units 540a, 540 b to higher power inlet connectors of the first and second dataservers 560 a, 560 b.

In at least one embodiment, the data center rack 500 additionallyincludes a low power peripheral device 580 physically coupled to therack enclosure 510. In at least one embodiment, the low power peripheraldevice is fixed within the rail system 525 using the one or morefasteners 528. In at least one embodiment, the low power peripheraldevice 580 is additionally electrically coupled to one or more of thepower distribution units 540 a, 540 b. In at least one embodiment, apower cord 590 couples a low power inlet connector of the powerdistribution unit 540 a to a low power inlet connector of the low powerperipheral device 580. In at least one embodiment, the low powerperipheral device 580 may comprise many different devices and remainwithin the scope of the disclosure. In at least one embodiment, the lowpower peripheral device 580 is a low power cooling device.

In at least one embodiment, the power distribution units, such as thepower distribution units 540 a, 540 b, are particularly advantageouswhen used in a rack enclosure 510 along with one or more data servers.In at least one embodiment, such a design reduces the overall airflowimpedance of the data center rack 500, and thus provides improvedcooling characteristics. In at least one embodiment, the rack mountpower distribution units 540 a, 540 b, in contrast to strip mountedpower distribution units, additionally allow the power supplies 572 ofthe first and second data servers 560 a, 560 b to be easily replaced,for example by sliding them out of the first and second data servers 560a, 560 b while the first and second data servers 560 a, 560 b remainfixed within the rail system 525. In at least one embodiment, forinstance in at least one typical strip mounted power distribution unitdesign, the strip mounted power distribution unit substantially impedesthe removal of the power supplies 572, whereas the rack mount powerdistribution units 540 a, 540 b do not.

Additional details for the power distribution unit, as well as a datacenter rack, as might be used in the data center 100 illustrated in FIG.1, are disclosed in U.S. patent application Ser. No. 16/798,790,entitled “POWER DISTRIBUTION UNIT,” filed Feb. 24, 2020, which iscommonly assigned herewith, the entirety of which is incorporated hereinby reference.

Aspects disclosed herein include:

A. A data center, the data center including: 1) a data center enclosure,the data center enclosure designed for a given supply of power (P_(s));and N independent coolable clusters of data center racks located withinthe data center enclosure, wherein N is at least two, and furtherwherein the N independent coolable clusters each have an ostensiblepower demand (P_(os)) approximately equal to P_(s)/N, and each of the Nindependent coolable clusters has a respective actual power demand(P_(ac)) adjustable at, above or below the ostensible power demand(P_(os)).

B. A method for cooling a data center, the method including: 1)providing a data center, the data center including: a) a data centerenclosure, the data center enclosure designed for a given supply ofpower (P_(s)); and b) N independent coolable clusters of data centerracks located within the data center enclosure, wherein N is at leasttwo, and further wherein the N independent coolable clusters each havean ostensible power demand (P_(os)) approximately equal to P_(s)/N, andeach of the N independent coolable clusters has a respective actualpower demand (P_(ac)) adjustable at, above or below the ostensible powerdemand (P_(os)); and 2) increasing an actual power demand (P_(ac)) of afirst of the N independent coolable clusters above the ostensible powerdemand (P_(os)), and decreasing an actual power demand (P_(ac)) of asecond of the N independent coolable clusters below the ostensible powerdemand (P_(os)).

C. A data center, the data center including: 1) a data center enclosure,the data center enclosure designed for a given supply of power (P_(s));and 2) N independent coolable clusters of data center racks locatedwithin the data center enclosure, wherein N is at least two, and furtherwherein at least one of a temperature, an airflow and a pressure isindependently adjustable across the N independent coolable clusters ofdata center racks.

D. A method for cooling a data center, the method including: 1)providing a data center, the data center including: a) a data centerenclosure, the data center enclosure designed for a given supply ofpower (P_(s)); and b) N independent coolable clusters of data centerracks located within the data center enclosure, wherein N is at leasttwo; and 2) independently adjusting at least one of a temperature, anairflow and a pressure across the N independent coolable clusters ofdata center racks.

Aspects A, B, C and D may have one or more of the following additionalelements in combination: Element 1: wherein the N independent coolableclusters are configured such that when a first of the N independentcoolable clusters has its actual power demand (P_(ac)) above theostensible power demand (P_(os)), a second of the N independent coolableclusters has its actual power demand (P_(ac)) below the ostensible powerdemand (P_(os)) in order to keep a sum of the actual power demands(P_(ac)) for the N independent coolable clusters at or below the givensupply of power (P_(s)). Element 2: wherein the data center enclosure isdesigned for a given cooling capacity (CC), and wherein the Nindependent coolable clusters each have a respective ostensible coolingcapacity (CC_(os)) approximately equal to CC/N, and further wherein thefirst of the N independent coolable clusters is configured such thatwhen its actual power demand (P_(ac)) is above the ostensible powerdemand (P_(os)) its actual cooling capacity (CC_(ac)) is increased aboveits ostensible cooling capacity (CC_(os)). Element 3: wherein the secondof the N independent coolable clusters is configured such that when itsactual power demand (P_(ac)) is below the ostensible power demand(P_(os)) its actual cooling capacity (CC_(ac)) is decreased below theostensible cooling capacity (CC_(os)). Element 4: wherein the Nindependent coolable clusters each include a multimode cooling system,the multimode cooling system providing the ability to increase ordecrease the respective actual cooling capacities (CC_(ac)) of the Nindependent coolable clusters above or below their ostensible coolingcapacities (CC_(os)). Element 5: wherein the multimode cooling system ofeach of the N independent coolable clusters comprises an air-basedcooling subsystem and a liquid-based cooling subsystem. Element 6:wherein the data center enclosure includes a raised floor for providingairflow to the N independent coolable clusters. Element 7: wherein N isat least four, and each of the at least four independent coolableclusters includes at least 8 data center racks. Element 8: wherein eachof the data center racks within the N independent coolable clustersincludes one or more data servers coupled to a respective powerdistribution unit. Element 9: wherein one or more of the powerdistribution units includes: a power distribution unit enclosure havinga connector surface; one or more low power inlet connectors extendingthrough the connector surface; and one or more higher power inletconnectors extending through the connector surface, wherein a ratio of anumber of the one or more higher power inlet connectors to a number ofthe one or more low power inlet connectors is at least 1:1. Element 10:wherein the data center enclosure is designed for a given coolingcapacity (CC), and wherein the N independent coolable clusters each havea respective ostensible cooling capacity (CC_(os)) approximately equalto CC/N, and further wherein each of the N independent coolable clustersare configured such that their actual cooling capacity (CC_(ac)) may beincreased above or decreased below their ostensible cooling capacity(CC_(os)). Element 11: wherein the multimode cooling system of each ofthe N independent coolable clusters comprises an air-based coolingsubsystem that at least one of the temperature, the airflow or thepressure may be adjusted, and a liquid-based cooling subsystem. Element12: wherein the liquid-based cooling subsystem is a cold plate singlephase cooling subsystem, a cold plate two phase cooling subsystem, or animmersion cooling subsystem. Element 13: wherein the N independentcoolable clusters each have an ostensible power demand (P_(os))approximately equal to P_(s)/N, and each of the N independent coolableclusters has a respective actual power demand (P_(ac)) adjustable at,above or below the ostensible power demand (P_(os)). Element 14: furtherincluding including keeping a sum of the actual power demands (P_(ac))for the N independent coolable clusters at or below the given supply ofpower (P_(s)) when increasing and decreasing. Element 15: wherein thedata center enclosure is designed for a given cooling capacity (CC), andwherein the N independent coolable clusters each have an ostensiblecooling capacity (CC_(os)), and further including increasing an actualcooling capacity (CC_(ac)) of the first of the N independent coolableclusters above its ostensible cooling capacity (CC_(os)) when its actualpower demand (P_(ac)) is increased above the ostensible power demand(P_(os)). Element 16: further including decreasing an actual coolingcapacity (CC_(ac)) of the second of the N independent coolable clustersbelow the ostensible cooling capacity (CC_(os)) when its actual powerdemand (P_(ac)) is decreased below the ostensible power demand (P_(os)).Element 17: wherein the N independent coolable clusters each include amultimode cooling system, and further wherein the increasing anddecreasing of the respective actual cooling capacities (CC_(ac))includes increasing and decreasing the respective actual coolingcapacities (CC_(ac)) using the multimode cooling systems. Element 18:wherein the data center enclosure is designed for a given coolingcapacity (CC), and wherein the N independent coolable clusters each havea respective ostensible cooling capacity (CC_(os)) approximately equalto CC/N, and further wherein each of the N independent coolable clustershas a respective ostensible cooling capacity (CC_(os)), and furtherincluding adjusting an actual cooling capacity (CC_(ac)) of one or moreof the N independent coolable clusters relative to their ostensiblecooling capacity (CC_(os)). Element 19: wherein adjusting the actualcooling capacity (CC_(ac)) includes increasing or decreasing the actualcooling capacity (CC_(ac)). Element 20: wherein the N independentcoolable clusters each include a multimode cooling system, and furtherincluding adjusting the actual cooling capacity (CC_(ac)) of the one ormore of the N independent coolable clusters using the multimode coolingsystem. Element 21: wherein the N independent coolable clusters eachhave an ostensible power demand (P_(os)) approximately equal to P_(s)/N,and further including adjusting an actual power demand (P_(ac)) of oneor more of the N independent coolable above or below the ostensiblepower demand (P_(os)).

Those skilled in the art to which this application relates willappreciate that other and further additions, deletions, substitutionsand modifications may be made to the described embodiments.

What is claimed is:
 1. A data center, comprising: a data centerenclosure, the data center enclosure designed for a given supply ofpower (P_(s)); and N independent coolable clusters of data center rackslocated within the data center enclosure, wherein N is at least two, andfurther wherein the N independent coolable clusters each have anostensible power demand (P_(os)) approximately equal to P_(s)/N, andeach of the N independent coolable clusters has a respective actualpower demand (P_(ac)) adjustable at, above or below the ostensible powerdemand (P_(os)).
 2. The data center as recited in claim 1, wherein the Nindependent coolable clusters are configured such that when a first ofthe N independent coolable clusters has its actual power demand (P_(ac))above the ostensible power demand (P_(os)), a second of the Nindependent coolable clusters has its actual power demand (P_(ac)) belowthe ostensible power demand (P_(os)) in order to keep a sum of theactual power demands (P_(ac)) for the N independent coolable clusters ator below the given supply of power (P_(s)).
 3. The data center asrecited in claim 2, wherein the data center enclosure is designed for agiven cooling capacity (CC), and wherein the N independent coolableclusters each have a respective ostensible cooling capacity (CC_(os))approximately equal to CC/N, and further wherein the first of the Nindependent coolable clusters is configured such that when its actualpower demand (P_(ac)) is above the ostensible power demand (P_(os)) itsactual cooling capacity (CC_(ac)) is increased above its ostensiblecooling capacity (CC_(os)).
 4. The data center as recited in claim 3,further wherein the second of the N independent coolable clusters isconfigured such that when its actual power demand (P_(ac)) is below theostensible power demand (P_(os)) its actual cooling capacity (CC_(ac))is decreased below the ostensible cooling capacity (CC_(os)).
 5. Thedata center as recited in claim 3, wherein the N independent coolableclusters each include a multimode cooling system, the multimode coolingsystem providing the ability to increase or decrease the respectiveactual cooling capacities (CC_(ac)) of the N independent coolableclusters above or below their ostensible cooling capacities (CC_(os)).6. The data center as recited in claim 5, wherein the multimode coolingsystem of each of the N independent coolable clusters comprises anair-based cooling subsystem and a liquid-based cooling subsystem.
 7. Thedata center as recited in claim 1, wherein the data center enclosureincludes a raised floor for providing airflow to the N independentcoolable clusters.
 8. The data center as recited in claim 1, wherein Nis at least four, and each of the at least six independent coolableclusters includes at least 8 data center racks.
 9. The data center asrecited in claim 1, wherein each of the data center racks within the Nindependent coolable clusters includes one or more data servers coupledto a respective power distribution unit.
 10. The data center as recitedin claim 9, wherein one or more of the power distribution unitsincludes: a power distribution unit enclosure having a connectorsurface; one or more low power inlet connectors extending through theconnector surface; and one or more higher power inlet connectorsextending through the connector surface, wherein a ratio of a number ofthe one or more higher power inlet connectors to a number of the one ormore low power inlet connectors is at least 1:1.
 11. A method forcooling a data center, comprising: providing a data center, the datacenter including: a data center enclosure, the data center enclosuredesigned for a given supply of power (P_(s)); and N independent coolableclusters of data center racks located within the data center enclosure,wherein N is at least two, and further wherein the N independentcoolable clusters each have an ostensible power demand (P_(os))approximately equal to P_(s)/N, and each of the N independent coolableclusters has a respective actual power demand (P_(ac)) adjustable at,above or below the ostensible power demand (P_(os)); and increasing anactual power demand (P_(ac)) of a first of the N independent coolableclusters above the ostensible power demand (P_(os)), and decreasing anactual power demand (P_(ac)) of a second of the N independent coolableclusters below the ostensible power demand (P_(os)).
 12. The method asrecited in claim 11, further including keeping a sum of the actual powerdemands (P_(ac)) for the N independent coolable clusters at or below thegiven supply of power (P_(s)) when increasing and decreasing.
 13. Themethod as recited in claim 12, wherein the data center enclosure isdesigned for a given cooling capacity (CC), and wherein the Nindependent coolable clusters each have an ostensible cooling capacity(CC_(os)), and further including increasing an actual cooling capacity(CC_(ac)) of the first of the N independent coolable clusters above itsostensible cooling capacity (CC_(os)) when its actual power demand(P_(ac)) is increased above the ostensible power demand (P_(os)). 14.The method as recited in claim 13, further including decreasing anactual cooling capacity (CC_(ac)) of the second of the N independentcoolable clusters below the ostensible cooling capacity (CC_(os)) whenits actual power demand (P_(ac)) is decreased below the ostensible powerdemand (P_(os)).
 15. The method as recited in claim 14, wherein the Nindependent coolable clusters each include a multimode cooling system,and further wherein the increasing and decreasing of the respectiveactual cooling capacities (CC_(ac)) includes increasing and decreasingthe respective actual cooling capacities (CC_(ac)) using the multimodecooling systems.
 16. The method as recited in claim 15, wherein themultimode cooling system of each of the N independent coolable clusterscomprises an air-based cooling subsystem and a liquid-based coolingsubsystem.
 17. The method as recited in claim 11, wherein the datacenter enclosure includes a raised floor for providing airflow to the Nindependent coolable clusters.
 18. The method as recited in claim 11,wherein N is at least six, and each of the at least six independentcoolable clusters includes at least 12 data center racks.
 19. The methodas recited in claim 11, wherein each of the data center racks within theN independent coolable clusters includes one or more data serverscoupled to a respective power distribution unit.
 20. The method asrecited in claim 19, wherein one or more of the respective powerdistribution units includes: a power distribution unit enclosure havinga connector surface; one or more low power inlet connectors extendingthrough the connector surface; and one or more higher power inletconnectors extending through the connector surface, wherein a ratio of anumber of the one or more higher power inlet connectors to a number ofthe one or more low power inlet connectors is at least 1:1.